Some computing systems host an application repository or other type of digital distribution platform for delivering computing software (e.g., applications) to computing devices. The computing system may distribute updated versions of an application in stages as part of a multi-staged rollout. In a multi-staged rollout, the computing system may distribute the updated version of the application to a limited group of computing devices in a test pool during a first stage so as to identify any major errors or bugs in the application. After fixing any errors identified in the first stage, the computing system may distribute the updated version of the application to a broader group of computing devices (e.g., any computing device requesting the application) during a second, subsequent stage of the staged rollout. In some cases, an error in the updated version of the application may only affect certain types of computing devices; such errors may go undetected during the first stage of a rollout if the group of computing devices being tested in the first stage do not sufficiently represent the types of computing devices that are affected by the error.